Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(clients): support util functions to consume response streams #3795

Closed
wants to merge 5 commits into from

Conversation

AllanZhengYP
Copy link
Contributor

@AllanZhengYP AllanZhengYP commented Jul 12, 2022

Issue

Associated to smithy-lang/smithy-typescript#571
Resolves: #1877

Description

Now if the deserialized object has streaming blob member(SSDK's input or SDK's output), customers can tranform the stream into string by

const body = response.Body;
const str = await body.transformToString("utf-8");

Or get the binary data by:

const binaryData: Uint8Array = await resp.Body.transformToByteArray();
// using buffer
const buf: Buffer = Buffer.from(await resp.Body.transformToByteArray());

Or transform to runtime-agnostic WebStream by:

// browsers and Node.js 16.5+
const webstream: ReadableStream = resp.Body.transformToWebStream();
const data = await webstream.json();
const buf = await webstream.arrayBuffer();

The output stream payload's interface is still assignable to the stream payload shape in the input, so this will work:

  new PutObjectCommand({ Body: bucket1Response.Body, Bucket: "Bucket2" });

At the same time, the payload stream is still an instance of the runtime-specific stream implementation, i.e. Stream.Readable in Node.js, ReadableStream in browsers and Blob in React Native. So the code like below will still work:

if (resp.Body instance of ReadableStream) {
  // Handle web stream
}

Note: since the streaming data can only be read once, and simplify the concurrency model, any of the transform methods to the stream can only be called once.

Testing

Unit test;
Integration test(PR to follow up)


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@AllanZhengYP AllanZhengYP force-pushed the stream-improvement-serde branch from 44c4d01 to 553b0db Compare July 13, 2022 17:59
@AllanZhengYP AllanZhengYP force-pushed the stream-improvement-serde branch 4 times, most recently from 64f8c55 to 2bd12ea Compare August 31, 2022 08:06
@kuhe
Copy link
Contributor

kuhe commented Sep 23, 2022

continued in #3977

@kuhe kuhe closed this Sep 23, 2022
@github-actions
Copy link

github-actions bot commented Oct 8, 2022

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 8, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

S3.GetObject no longer returns the result as a string
3 participants